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Claims : 

1. (Amended) A method for image data processing^ comprising: 
separating object data of an image data set (object data 

set) representing an image foreground from a data set (background 
data set) representing an image background^ and 

altering a background to be recorded by the camera at a 
frequency tuned to a recording frequency of the recording camera. 

2. (Trended) The method according to Claim 1, wherein the 
altering of the background occurs by switching between two 
states. 

3- (Amended) The method according to Claim 1, wherein the 
altering of the background occurs gradually between two states 
with multiple intermediate states. 

4. (Amended) The method according to Claim 1, wherein the 
color and/or brightness and/or saturation of the background is 
altered . 

5. (Amended) The method according to Claim If wherein the 
recording frequency of the camera is a multiple of the playback 
frequency. 

6. The method according to Claim 5, wherein the recording 
frequency is double the typical television image frequency. 

7. (Amended) The method according to Claim 1, wherein a 
comparison of the data sets of camera images recorded at a 
different instant within a replay frequency is performed to 
establish the assignment of an element of a camera image data set 
to the object data set or the background data set. 
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8. The method according to Claim 1, wherein the comparison 
includes a subtraction of elements of identical coordinates or 
regions of the camera image data sets. 

9. (Amended) The method according to Claim 7, wherein the 
elements of the camera image data sets compared with one another 
are pixels. 

10. (Amended) The method according to Claim 8, wherein the 
elements of the camera image data sets compared with one another 
are pixel regions. 

11. (Amended) The method according to Claim 1, wherein methods 
of artificial intelligence, such as neuronal nets, are used for 
recognition and assignment of the elements of the camera image 
data sets. 

12. (Amended) The method according to Claim 1, wherein the 
elements of the camera image data set identified as associated 
with the background data set are replaced by elements of 
corresponding coordinates or regions of a data set of a 
replacement background. 

13. (Amended) A camera recording system Claim 1, comprising: 

at least one camera for recording an object located in front 
of a background, with an alteration of the background occurring 
at a frequency tuned to a recording frequency of the recording 
camera • 

14. The camera recording system according to Claim 13, 
wherein a film-like or sheet-like element inf luenceable in an 
optically defined way is provided as a background. 



10 
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15. The camera recording system according to Claim 14, 
wherein the background element is an electroluminescent or 
electrochromatic flat element - 



11 
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Claims : 



1. A method for image data processing, 

wherein object data of an image data set (object 
data set) representing an image foreground is separated from a 
data set {background data set) representing an image background. 



wherein a background to be recorded by the camera 
is altered at a frequency tuned to a recording frequency of the 
recording camera. 

2. The method according to Claim 1, wherein the 
alteration of the background occurs by switching between two 
states . 



3. The method according to Claim 1, wherein the 
alteration of the background occurs gradually between two states 
with multiple intermediate states. 

4 . The method according to one of Claims 1 to 3 ^^^^^r 
wherein the color and/or brightness and/or saturation of the 
background is altered. 

5 . The method according to one of Claims 1 to 4 ^^^^fif 
wherein the recording frequency of the camera is a multiple of 
the playback frequency. 



6. The method according to Claim 5, wherein the recording 
frequency is double the typical television image frequency. 
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7 . The method according to one of Claims 1 — to 0 iS^^HsIf 
wherein a comparison of the data sets of camera images recorded 
at a different instant within a replay frequency is performed to 
establish the assignment of an element of a camera image data set 
to the object data set or the background data set. 

8. The method according to Claim 7, wherein the comparison 
includes a subtraction of elements of identical coordinates or 
regions of the camera image data sets. 

9. The method according to Claim 7 or 0 ^^^^M' wherein the 
elements of the camera image data sets compared with one another 
are pixels. 

10. The method according to Claim 0 or D S^^S' wherein the 
elements of the camera image data sets compared with one another 
are pixel regions. 

11. The method according to one of Claims 1 to C ^^B^S' 
wherein methods of artificial intelligence, such as neuronal 
nets, are used for recognition and assignment of the elements of 
the camera image data sets. 

12. The method according to one of Claima G to 11 ^^^M/ 
wherein the elements of the camera image data set identified as 
associated with the background data set are replaced by elements 
of corresponding coordinates or regions of a data set of a 
replacement background . 

13. A camera recording syste m, particulaijly for peilorming the 
method according Lo one of Claims 1 to 12 ^^^^Mr ^^^^^^Bl 

having at least one camera for recording an object located 
in front of a background, with an alteration of the background 
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occurring at a frequency tuned to a recording frequency of the 
recording camera. 

14. The camera recording system according to Claim 13, 
wherein a film-like or sheet-like element inf luenceable in an 
optically defined way is provided as a background. 

15. The camera recording system according to Claim 14, 
wherein the background element is an electroluminescent or 
electrochromatic flat element- 
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METHOD FOR IMAGE DATA PROCESSING AND CAMERA RECORDING SYSTEM 

The present invention relates to a method for image data 
processing in which object data of an image data set representing 
an image foreground is separated from a data set representing an 
image background^ as well as a camera recording system,, which is 
particularly suitable for performing a method of this type. 

The blue screen method is known from television technology^ in 
which a surface forming the background of the setting to be 
recorded is implemented as a monochrome, particularly blue, 
background. The recording of an object in front of this 
background is first recorded by a television camera at an image 
frequency of, normally, 50 to 60 half-frames per second, while in 
parallel to this a background replacing the blue background of 
the first image, for example a weather map, is recorded by a 
second television camera. The two image sequences recorded in 
parallel are superimposed in synchronization on one another in a 
mixer and the blue background area is replaced by the image 
recorded with the second television camera. 

In the blue screen method, two principle methods are 
nj differentiated, the first of which is based on a definition of 

the chromaticity (the color representing the background is 
defined as transparent) , while in the second method, which was 
commercially exploited particularly by the firm Ultima tte, the 
background is defined via the consistency of the brightness 
and/or saturation of the color space of the background. 

The advantage of the known methods is that the processing of the 
analog image data is relatively simple and occurs in real time. 
It is a disadvantage that the typically blue, but also frequently 
green, background has to be very well illuminated, since the 
color or brightness and/or saturation of the color, respectively, 
must be very uniform and of high quality in the image sequence 
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recorded, which may only be achieved by excellent illumination of 
the colored surface representing the background. This 
illumination is, however, connected with a very high outlay, 
since the typical lighting time for a recording studio is 
approximately one week, or often even more. Not considering the 
man time connected with this, a large amount of power is consumed 
and unpleasantly large amounts of heat are produced for those 
working in the vicinity of the recording. Furthermore, it is 
disadvantageous that shadows and/or reflections must be avoided, 
which makes the lighting even more difficult. Reflections may not 
be corrected with the known methods, and shadows only with 
difficulty. 

However, according to the present invention a method for image 
data processing having the features of Claim 1 and a camera 
recording system having the features of Claim 8 is suggested. 

The present invention is based on the idea that the background is 
altered in a defined way at a frequency tuned to the recording 
frequency of the recording camera. Tuning is understood here to 
mean that the frequency of the background alteration has a known 
ratio to the recording frequency. This may be an integer multiple 
of the recording frequency, but could also be any desired 
multiple. The recording frequency is preferably a multiple of the 
playback frequency, for example double the typical television 
image frequency, i.e., approximately 100 to 120 half -frames per 
second. In principle, the method according to the present 
invention functions, as will be described in the following, at 
any desired recording frequency up to several thousand images per 
second, with the alteration frequency of the background tuned to 
this. A requirement is that the content (foreground) of two 
images recorded in sequence does not alter or at least alters 
only insignificantly, since every second image of the image 
sequence is used as a reference image. 
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The method according to the present invention opens up the 
possibility of using digital cameras and of the utilization of 
their technical advantages. The recording of an image sequence 
at, for example, double frequency with a background changing in a 
defined way corresponding to this frequency leads to a sequence 
of image pairs, with each two images forming an image pair (at 
the typical movement speeds of recorded objects) not or barely 
able to be differentiated from one another except for the 
background, which alters in a defined way from one recording to 
the next. One of the two images of an image pair may thus be used 
as a reference image for the respective other image. 

Using the advantages of digital recording technology-each 
recorded pixel of an image may be directly compared with the 
corresponding pixel of the following image-the two images of an 
image pair are compared with one another, in that, for example, 
each pixel of the reference image is subtracted from the 
corresponding pixel of the other image. While, as a result of the 
comparison, a value of zero or almost zero results for the 
foreground of the object forming the recording if there is no 
alteration (non-moving or slowly moving object) and an arbitrary 
value not equal to zero results for a slight alteration (rapidly 
moving object), the comparison of the background surfaces results 
in a predetermined defined value, since the background was 
purposely altered in a defined way from the recording of the one 
image to the next image. The alteration of the background may be 
performed by switching between two states or by gradual 
continuous oscillation, for example sinusoid, between the two 
states . 

In order to now separate the foreground of the object forming the 
recording from the background, to "clip it out," all pixels to 
which a defined content was assigned during the comparison of the 
two images forming an image pair are suppressed and possibly 
replaced by the content of an image recorded in parallel (for 
example the weather map) .. This second image may have any desired 

3 
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source. Instead of the image recorded in parallel as described,, a 
single image (fixed image) may also be used as a background or, 
for example, even a computer-generated image and/or image 
sequence . 

In a preferred embodiment of the invention, to determine the 
assignment of an element of a camera image data set to the object 
data set or the background data set, a comparison of the data 
sets of camera images recorded at a different instant within a 
playback frequency is performed • If the recording frequency is, 
for example, three times the later playback frequency, then 
during one playback frequency, i.e., within a period 
corresponding to the playback frequency, three camera images are 
recorded. These three images are compared with one another to 
identify any desired element in the camera image as an object or 
background. The comparison preferably occurs in pairs. Thus, for 
example, the first camera image of this sequence of three images 
is first compared with the second image and then with the third 
image, and finally the second image is compared with the third 
image. In order to design the method more economically, the 
comparison may be performed in steps, with sequential comparison 
steps only performed if the prior method steps have not yet 
provided an unequivocal result, i.e., have not yet allowed 
unequivocal assignment of the camera image element as object or 
background. 

In a particularly preferred embodiment, the comparison is a 
subtraction of elements of identical coordinates or regions of 
the camera image data sets, with the elements of the camera image 
data sets to be compared with one another able to be single 
pixels or even pixel regions. A comparison of pixel regions has 
been shown to be particularly advantageous, since the method may 
then be performed more rapidly. It suggests itself in this case 
to first select larger pixel regions, in order possibly to 
subdivide these into subregions step-by-step. The method 
according to the present invention also opens up the possibility 

4 
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of comparing not only two-dimensional pixel regions with one 
another, but also n-dimensional pixel regions. 

As described above, the method according to the present invention 
may be performed at any desired recording frequency up to several 
thousand images per second. The higher the recording frequency 
is, the larger the number of images having different background 
states (particularly background brightnesses) recorded within one 
playback period, which causes the comparison possibilities, i.e., 
the permutations of comparison pairs, to increase significantly. 
In this way, a better foundation for the assignment of a camera 
image element to the background data set or the object data set 
is given. 

In addition, the present invention opens up the possibility of 
evaluating the reflection behavior of objects in the camera image 
at a high recording frequency and gradual background alteration 
and drawing conclusions about the refractive index and the 
material of the object from this. This, in turn, allows the 
adjustment of the relevant data set elements of the object data 
set to the new replacement background, which replaces the 
original recording background. In contrast to the methods known 
from the related art, not only is correction of reflections (and 
shadows) possible according to the present invention, but also 
their evaluation, in order to allow the recognition of the 
affiliation as object or background of an image element or the 
adjustment of the image and/or the object to a background used. 
In this way, shadows and/or reflections on the object and/or the 
background may be recognized according to the present invention 
and, depending on the application or the desire of the user, 
suppressed or adjusted and processed in another way. 

The method according to the present invention is preferably 
performed using a neuronal net or other methods of artificial 
intelligence (AI) . In this way, particularly good recognition and 



wo 01/06766 PCT/EPOO/06827 

assignment of the image elements as object, background, 
reflection, shadow, etc. is achieved. 

Finally, the present invention opens up the possibility of 
investigating the camera image data sets not only for an 
assignment in the two-dimensional range (object and background 
affiliation) , but also to obtain further information about the 
object and/or the background, such as the speed, among other 
things. This in turn allows conclusions about three-dimensional 
data of the camera image data set (distance from the camera 
etc.), so that the present invention is suitable not only for 
two-dimensional, but also for multidimensional image data 
processing. 

Furthermore, it has been shown to be advantageous to alter the 
recording background only partially during the recording. 

To perform the method according to the present invention, film- 
like or sheet-like elements are preferably used, which may be 
optically influenced very rapidly in a targeted way, i.e., their 
color and/or brightness and/or saturation may be switched from a 
first preset value to a second preset value in a very short time. 
For this purpose, electroluminescent or electrochromatic flat 
elements are particularly suitable, as they are already known 
from the related art. 

Further advantages and embodiments of the present invention 
result from the description and the attached drawing. 

It is obvious that the features previously mentioned and 
described in the following are usable not only in the respective 
combination indicated, but also in other combinations or alone, 
without leaving the framework of the present invention. 
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The present invention is schematically illustrated in the drawing 
with reference to an exemplary embodiment and is described in 
detail in the following with reference to the drawing . 

Fig. 1 shows a block diagram to illustrate the present 
invention . 

Fig. 2 shows a block diagram to illustrate an evaluation unit 
to be used in the present invention. 

The central component of the invention is the recording of one or 
more objects (objects 1 to m) by one or more cameras (cameras 1 
to n) , the objects being located in front of a background. The 
background is, according to the present invention, inf luenceable 
and/or alterable at a high frequency. The alteration frequency is 
tuned to a recording frequency of the recording camera (s) in this 
case. The alteration of the background particularly occurs in 
regard to its color and/or brightness and/or saturation. 

The inf luenceable parameters of the background are thus, besides 
the alteration frequency, the color, the brightness, and/or the 
saturation. In the camera (s), the inf luenceable parameter is, 
besides the typical camera parameters, the recording frequency, 
possibly restricted to a definable image region x*y. These 
parameters are controlled by the evaluation unit, with direct 
control, particularly for adjustment of the camera (s) to the 
background, also being possible. 

Objects 1 to m included by the background, i.e., surrounded by 
the background, are recorded with cameras 1 to n, the recording 
frequency of the cameras being a multiple of the later playback 
frequency. As a result, Az camera images from n cameras are 
provided per time unit z, which are supplied to the evaluation 
unit illustrated in more detail in Fig. 2. 
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The evaluation unit illustrated in Fig. 2 is^, for each of the 1 
to n cameras^, its own camera-specific evaluation unit is 
provided, which are each supplied the data of the Az camera 
images. In consideration of the additional parameters, as they 
were described previously, also supplied to each camera-specific 
evaluation unit, in each camera-specific evaluation unit the 
object (s) is/are separated from a background, in that an 
identification and an assignment of individual camera image 
elements (pixels or pixel regions) to object" or ^''background'' 
occurs. After the assignment is performed, the image elements 
assigned to the background are removed and replaced by image 
elements of a replacement background • In simple applications, 
this replacement background is a static, unchanging image, for 
example the weather map. However, more complex applications are 
also possible and are made easier by the present invention, such 
as the overlay of separately recorded images or moving images 
generated by a computer as a background during studio recordings. 
The cycle of the identification was already described above. 

The image data produced by the individual camera-specific 
evaluation units is then supplied to a central recording-instant- 
specific evaluation unit, which includes a buffer for evaluating 
multiple time units and is used for synchronization and tuning of 
the data supplied by the camera-specific evaluation units. The 
usage of multiple cameras allows the generation of image data 
which is suitable for playback observable by an observer from 
different perspectives as three-dimensional ."^ A recording and 
playback method of this type is, for example, described in German 
Patent Application 199 13 853.2. 

The data produced by the central recording-instant-specific 
evaluation unit is output from the evaluation unit, so that a 
single image (frame) is present as a result (cf. Fig. 1), in 
which the background and the object are separated from one 
another. A typical playback frequency of the frame produced is 
approximately 25 to 30 sec"^ (television image frequency) . 

8 
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Claims : 

1. A method for image data processing, 

wherein object data of an image data set (object data set) 
representing an image foreground is separated from a data set 
(background data set) representing an image background, wherein a 
background to be recorded by the camera is altered at a frequency 
tuned to a recording frequency of the recording camera, 

2. The method according to Claim 1, 

wherein the alteration of the background occurs by switching 
between two states . 

3. The method according to Claim 1, wherein the alteration of 
the background occurs gradually between two states with multiple 
intermediate states . 

4. The method according to one of Claims 1 to 3, 
wherein the color and/or brightness and/or saturation of the 
background is altered. 

5. The method according to one of Claims 1 to 4, 

wherein the recording frequency of the camera is a multiple of 
the playback frequency. 

6. The method according to Claim 5, 

wherein the recording frequency is double the typical television 
image frequency . 

7. The method according to one of Claims 1 to 6, 

wherein a comparison of the data sets of camera images recorded 
at a different instant within a replay frequency is performed to 
establish the assignment of an element of a camera image data set 
to the object data set or the background data set. 

8. The method according to Claim 7, 
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wherein the comparison includes a subtraction of elements of 
identical coordinates or regions of the camera image data sets. 

9. The method according to Claim 7 or 8, 

wherein the elements of the camera image data sets compared with 
one another are pixels. 

10. The method according to Claim 8 or 9, 

wherein the elements of the camera image data sets compared with 
one another are pixel regions. 

11. The method according to one of Claims 1 to 6, 
wherein methods of artificial intelligence, such as neuronal 
nets, are used for recognition and assignment of the elements of 
the camera image data sets. 

12. The method according to one of Claims 6 to 11, 

wherein the elements of the camera image data set identified as 
associated with the background data set are replaced by elements 
of corresponding coordinates or regions of a data set of a 
replacement background. 

13. A camera recording system, particularly for performing the 
method according to one of Claims 1 to 12, having at least one 
camera for recording an object located in front of a background, 
with an alteration of the background occurring at a frequency 
tuned to a recording frequency of the recording camera. 

14. The camera recording system according to Claim 13, 
wherein a film-like or sheet-like element inf luenceable in an 
optically defined way is provided as a background. 

15. The camera recording system according to Claim 14, 
wherein the background element is an electroluminescent or 
electrochromatic flat element. 
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ABSTRACT 



The invention relates to a method for image data processing, 
whereby object data of an image data set representing an image 
foreground (object data set) is separated from data set 
representing an image background (background data set) , whereby a 
background to be recorded by a camera is changed into a frequency 
matching the recording frequency of the recording camera. The 
background is changed, for instance, by switching between two 
states or by gradually switching between two states having a 
plurality of intermediate states. The invention also relates to a 
camera recording system, especially for implementing the inventive 
method, comprising at least one camera for recording an object 
located in front of a background, whereby the background is 
changed into a frequency matching the recording frequency of the 
recording camera. 
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Attorney's Docket No.: 7195 
DECLARATION AND POWER OF ATTORNEY - ORIGINAL APPLICATION 



As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated 
below beneath my name: 

I believe I am the original, first and sole inventor (if only one 
name is listed below) or an original, first and joint inventor (if 
plural names are listed below) of the subject matter which is 
claimed and for which Letters Patent is sought on the invention 
entitled 

METHOD FOR IMAGE DATA PROCESSING AND 
CAMERA RECORDING SYSTEM 



is attached hereto 

X was filed on 17 Julv 2000 

as Application S.N. PCT/EPOO/06827 
and was amended on 

I hereby state that I have reviewed and understand the contents 
of the above -identified specification, including the claims, as 
amended by an amendment referred to above. 

I acknowledge the duty to disclose information which is material 
to the examination of this application in accordance with Title 
37, Code of Federal Regulations, §1.56 (a). 

I hereby claim foreign priority benefits under Title 35, United 
States Code, §119 (a) - (d) or §365 (b) of any foreign application (s) 
for patent or inventor's certificate, or §365 (a) of any PCT 
International Application which designated at least one country 
other than the United States, listed below and have also 
identified below any foreign application for patent or invention 
certificate, or a PCT International application having a filing 
date before that of the application on which priority is claimed. 

Prior Foreign Application (s) Priority Claimed 

DE 199 32 663>0 Germany 15 Julv 1999 X 

(Number) (Country) (Day /Month/ Year Filed) YES NO 



the specification of which 
(check one) 



(Number) 



(Country) (Day/Month/Year Filed) YES NO 



I hereby claim the benefit under 35 U.S.C. §119 (e) of any United 
States provisional application (s) listed below: 



(Application Ser. No.) 



(Filing Date) 



(Application Ser. No.) 



(Filing Date) 



I hereby claim the benefit under Title 35, United States Code, 
§120 of any United States application (s) , or §365 (c) of any PCT 
International application designating the United States, listed 
below and, insofar as the subject matter of each of the claims of 
this application is not disclosed in the prior United States 
application or PCT International application in the manner 
provided by the first paragraph of Title 35, United States Code, 
§112, I acknowledge the duty to disclose material information as 
defined in Title 37, Code of Federal Regulations, §1.56 (a) which 
occurred between the filing date of the prior application and the 
national or PCT international filing date of this application. 



POWER OF ATTORNEY: As a named inventor, I hereby appoint the 
following attorney (s) and/or agent (s) to prosecute this 
application and transact all business in the Patent and Trademark 
Office connected therewith: 

B. Edward Shlesinger, Jr., Reg. No. 17,225 ; George A. Arkwright, 
Reg. No. 20,158; George A. Qarvey, Reg. No. 17,737 ; Josefino P. 
de Leon, Reg. No. 33,166 ; Terrence L. B. Brovm, Reg. No. 32^j585; 
Michael M. Zadrozny, Reg. No. 30,985 ; and Brian J. Marton, Reg. 
Agent 30,292; 

SEND CORRESPONDENCE TO: SHLESINGER, ARKWRIGHT & GAR VEY LLP 

3000 South Eads Stre et 
Arlington, Virginia 2220 2 

DIRECT TELEPHONE CALLS TO Josefino P. de L eon (703) 684-5600 

The undersigned hereby authorizes the U.S. attorney (s) or 
agent (s) named herein to accept and follow instructions from 
HOSSLE & KUDLEK as to any action to be taken in the Patent and 
Trademark Office regarding this application without direct 
communication between the U.S. attorney (s) or agent (s) and the 
undersigned. In the event of a change in the persons from whom 
instructions may be taken, the U.S. attorney (s) or agent (s) named 
herein will be so notified by the undersigned. 



(Application Ser. No.) 



(Filing Date) (Status) 

(patented, pending, abandoned) 



k I hereby declare that all statements made herein of my own 

knowledge are true and that all statements made on information 
and belief are believed to be true; and further that the 
statements were made with the knowledge that willful false 
statements and the like so made are punishable by fine or 
imprisonment, or both, under §1001 of Title 18 of the United 
States Code and that such willful false statements may jeopardize 
the validity of the application or any patent issuing thereon. 

, , _ Full name of sol. or flrtt^vjntor pKT^jBS ^^.^^ 

Inventor's signature X y^f ^/,^ ^ 

Residence Ann Wallariben 142. D-7Q565, gfcutitaart. Germany 

Citizenship_Germaii _ 

Post Office AHfSrg^ss same as above , 



Full name of second or /To^t/inv^to^^IT^ . _ 

Inventor's signature tC J/cA/ ^ (A fcXi^ Date K4^5'^*® ^ 

Residence Arm Wallaraben 142. D -7Q565 Stuttgart. Germany Jii^A. 

Cit i zenship_German_ 



Post Office a^^-K-agg gaTTie above 
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